Popular Comments
Recent Discussion
Epistemic status: This post aims at an ambitious target: improving intuitive understanding directly. The model for why this is worth trying is that I believe we are more bottlenecked by people having good intuitions guiding their research than, for example, by the ability of people to code and run evals.
Quite a few ideas in AI safety implicitly use assumptions about individuality that ultimately derive from human experience.
When we talk about AIs scheming, alignment faking or goal preservation, we imply there is something scheming or alignment faking or wanting to preserve its goals or escape the datacentre.
If the system in question were human, it would be quite clear what that individual system is. When you read about Reinhold Messner reaching the summit of Everest, you would be curious about...
(Other than the thoughts on the consequences of said idea) This idea largely seems like a rehash of https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators (and frankly, so does the three layer model, but that does go into more mechanistic territory and I think it complements simulator theory well)
This post was adapted from an internal doc I wrote at Wave.
Welcome to being a manager! Your time-management problem just got a lot harder.
As an IC, you can often get away with a very simple time-management strategy:
- Decide what your one most important thing is.
- Work on it until it’s done.
- GOTO 1
As a team lead, this isn’t going to work, because much more of your work is interrupt-driven or gets blocked for long periods of time. One-on-ones! Code reviews! Design reviews! Prioritization meetings! Project check-ins! All of these are subject to an external schedule rather than being the type of thing that you can push on in a single focused block until it’s done.
Being a team lead means three big changes for your time management:
-
You no longer have a
A key step in the classic argument for AI doom is instrumental convergence: the idea that agents with many different goals will end up pursuing the same few subgoals, which includes things like "gain as much power as possible".
If it wasn't for instrumental convergence, you might think that only AIs with very specific goals would try to take over the world. But instrumental convergence says it's the other way around: only AIs with very specific goals will refrain from taking over the world.
For idealised pure consequentialists -- agents that have an outcome they want to bring about, and do whatever they think will cause it -- some version of instrumental convergence seems surely true[1].
But what if we get AIs that aren't pure consequentialists, for example because they're...
Something entirely new occurred around March 26th, 2025. Following the release of OpenAI’s 4o image generation, a specific aesthetic didn’t just trend—it swept across the virtual landscape like a tidal wave. Scroll through timelines, and nearly every image, every meme, every shared moment seemed spontaneously re-rendered in the unmistakable style of Studio Ghibli. This wasn’t just another filter; it felt like a collective, joyful migration into an alternate visual reality.
But why? Why this specific style? And what deeper cognitive or technological threshold did we just cross? The Ghiblification wave wasn’t mere novelty; it was, I propose, the first widely experienced instance of successful reality transfer: the mapping of our complex, nuanced reality into a fundamentally different, yet equally coherent and emotionally resonant, representational framework.
And Ghibli, it turns out, was...
A thing that gave me creeping horror about the Ghiblification is that the I don't think the masses actually particularly understand Ghibli. And the result is an uneven simulacrum-mask that gives the impression of "rendered with love and care" without actually being so.
The Ghibli aesthetic is actually historically pretty important to me, and in particular important as a counterbalanacing force against, among other things, what I expect to happen by default with AI. Some things I like about Ghibli:
- The "cinematic lens" emphasizes a kind of "see everythi
Our community is not prepared for an AI crash. We're good at tracking new capability developments, but not as much the company financials. Currently, both OpenAI and Anthropic are losing $5 billion+ a year, while under threat of losing users to cheap LLMs.
A crash will weaken the labs. Funding-deprived and distracted, execs struggle to counter coordinated efforts to restrict their reckless actions. Journalists turn on tech darlings. Optimism makes way for mass outrage, for all the wasted money and reckless harms.
You may not think a crash is likely. But if it happens, we can turn the tide.
Preparing for a crash is our best bet.[1] But our community is poorly positioned to respond. Core people positioned themselves inside institutions – to advise on how to maybe make AI 'safe',...
Yes, I get you don’t just want to read about the problem but a potential solution.
The next post in this sequence will summarise the plan by those experienced organisers.
These organisers led one of the largest grassroots movements in recent history. That took years of coalition building, and so will building a new movement.
So they want to communicate the plan clearly, without inviting misinterpretations down the line. I myself rushed writing on new plans before (when I nuanced a press release put out by a time-pressed colleague at Stop AI). That...
I've been running meetups since 2019 in Kitchener-Waterloo. These were rationalist-adjacent from 2019-2021 (examples here) and then explicitly rationalist from 2022 onwards.
Here's a low-effort/stream of consciousness rundown of some meetups I ran in Q1 2025. Sometime late last year, I resolved to develop my meetup posts in such a way that they're more plug-and-play-able by other organizers who are interested in running meetups on the same topics. Below you'll find links to said meetup posts (which generally have an intro, required and supplemental readings, and discussion questions for sparking conversation—all free to take), and brief notes on how they went and how they can go better. Which is to say, this post might be kind of boring for non-organizers.
The Old Year and the New
The first meetup of...
If there are a lot of people for the very-low-context NY meetup, possibly at least one very-low-context meetup per quarter is worth doing, to see if that gets people in/back more?
Given that 'Seize control of the capital markets and use that unlimited funding to...' is on the critical path for a number of superintelligence doom scenarios, this paper might be interesting to the safety community.
IOSCO is an international body for coordinating capital market regulators, so this represents their collective view on the challenges presented by AI tech in the near to medium term.
https://www.iosco.org/library/pubdocs/pdf/IOSCOPD788.pdf
I think rationalists should consider taking more showers.
As Eliezer Yudkowsky once said, boredom makes us human. The childhoods of exceptional people often include excessive boredom as a trait that helped cultivate their genius:
A common theme in the biographies is that the area of study which would eventually give them fame came to them almost like a wild hallucination induced by overdosing on boredom. They would be overcome by an obsession arising from within.
Unfortunately, most people don't like boredom, and we now have little metal boxes and big metal boxes filled with bright displays that help distract us all the time, but there is still an effective way to induce boredom in a modern population: showering.
When you shower (or bathe, that also works), you usually are cut off...
"Stimulating" here is not quite the opposite of "boring". Many minds are used to said temperature changes, water assaults, and laborious motions, such that they still stimulate, but are easily ignored, leaving much space for thoughts. Showers are boring by consistency, despite stimulation.
PDF version. berkeleygenomics.org. Twitter thread. (Bluesky copy.)
Summary
The world will soon use human germline genomic engineering technology. The benefits will be enormous: Our children will be long-lived, will have strong and diverse capacities, and will be halfway to the end of all illness.
To quickly bring about this world and make it a good one, it has to be a world that is beneficial, or at least acceptable, to a great majority of people. What laws would make this world beneficial to most, and acceptable to approximately all? We'll have to chew on this question ongoingly.
Genomic Liberty is a proposal for one overarching principle, among others, to guide public policy and legislation around germline engineering. It asserts:
Parents have the right to freely choose the genomes of their children.
If upheld,...
Whether you do it by genetic engineering or surgically or through some other means is entirely beside the point. Genetic engineering isn't special.
I'm not especially distinguishing the methods, I'm mainly distinguishing whether it's being done to a living person. See my comment upthread https://www.lesswrong.com/posts/rxcGvPrQsqoCHndwG/the-principle-of-genomic-liberty?commentId=qnafba5dx6gwoFX4a
...We get genetic engineering by showing people that it is just another technology, and we can use it to do good and not evil, applying the same notions of good a
